Methods and apparatus for watermark outage detection

ABSTRACT

Methods, apparatus, systems and articles of manufacture are disclosed for watermark outage detection. Example methods include evaluating an onset time and duration of a detected watermark outage based on a model of expected watermark outages to determine whether the detected watermark outage corresponds to at least one of the expected watermark outages represented in the model. Example methods further include generating an alert in response to determining the detected watermark outage does not correspond to at least one of the expected watermark outages included in the model, and suppressing the alert in response to determining the detected watermark outage corresponds to the at least one of the expected watermark outages represented in the model.

RELATED APPLICATION

This patent arises from a continuation of U.S. patent application Ser.No. 16/051,091 (now U.S. Pat. No. 10,827,209), which was filed on Jul.31, 2018. U.S. patent application Ser. No. 16/051,091 is herebyincorporated herein by reference in its entirety. Priority to U.S.patent application Ser. No. 16/051,091 is hereby claimed.

FIELD OF THE DISCLOSURE

This disclosure relates generally to watermark encoding, and, moreparticularly, to methods and apparatus for watermark outage detection.

BACKGROUND

Some media monitoring applications embed watermarks into media signalsto enable subsequent detection of the media conveyed by the mediasignals by decoding the watermarks in a presented media signal. Forexample, a broadcasting entity (e.g., a radio broadcaster, a televisionbroadcaster, an internet streamer, etc.) may encode watermarks intomedia signals. A media monitoring entity may then detect the watermarksin the media signals during monitoring activity and accuratelydetermine, based on identification information associated with thewatermark, that the media (e.g., a television show, a film, a commercialetc.) corresponding to the media signals was presented. Reliable andconsistent encoding of the watermarks in the presented media signalsensures accurate metrics of exposure to the media corresponding to themedia signals.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment for monitoring ofwatermarked media signals in accordance with the teachings of thisdisclosure.

FIG. 2 is a block diagram of an example watermark outage monitor todetect watermark outages in accordance with the teachings of thisdisclosure.

FIGS. 3A-3B are flowcharts representative of machine readableinstructions that may be executed to implement the watermark outagemonitor of FIGS. 1 and 2 to generate and/or update a sequence model anda grouping model.

FIG. 4 is a flowchart representative of machine readable instructionswhich may be executed to implement the watermark outage monitor of FIGS.1 and 2 to analyze a detected watermark encoding outage and determine ifa watermark outage alert should be generated.

FIGS. 5A-5E represent example watermark detection plots for a portion ofan example data collection period.

FIG. 5F is a state diagram based on the data collection period of thewatermark detection plots of FIGS. 5A-5E.

FIG. 6A is an example outage duration plot including watermark outageshaving two example outage onset times that can be represented by twooutage clusters.

FIG. 6B is an example outage duration plot including watermark outageshaving three example outage onset times that are represented by threeoutage clusters.

FIG. 6C is an example outage duration plot depicting an outlierwatermark outage exceeding an example duration alarm threshold.

FIG. 7A is an example outage duration plot including watermark outageswith same onset times represented by more than one cluster.

FIG. 7B is an example outage duration plot representing the outage dataof FIG. 7A reorganized in an adjusted model.

FIG. 8 is a block diagram of an example processor platform structured toexecute the example machine readable instructions of FIGS. 3A-B and/or 4to implement the example watermark outage monitor of FIGS. 1-2.

The figures are not to scale. In general, the same reference numberswill be used throughout the drawing(s) and accompanying writtendescription to refer to the same or like parts.

DETAILED DESCRIPTION

Watermarking is a technique used to identify media such as televisionbroadcasts, radio broadcasts, advertisements (television and/or radio),downloaded media, streaming media, prepackaged media, etc. Existingwatermarking techniques identify media by encoding one or more codes(e.g., one or more watermarks) including media identifying informationand/or an identifier that may be mapped to media identifyinginformation, into an audio and/or video component. In some examples, theaudio or video component is selected to have a signal characteristicsufficient to hide the watermark. As used herein, the terms “code” or“watermark” are used interchangeably and are defined to mean anyidentification information (e.g., an identifier) that may be inserted orencoded in the audio or video of media (e.g., a program oradvertisement) for the purpose of identifying the media or for anotherpurpose such as tuning (e.g., a packet identifying header). As usedherein “media” refers to audio and/or visual (still or moving) contentand/or advertisements. Further, as used herein, the term “media”includes any type of content and/or advertisement delivered via any typeof distribution medium. Thus, media includes television programming oradvertisements, radio programming or advertisements, movies, web sites,streaming media, etc. To identify watermarked media, the watermark(s)are extracted and used to access a table of reference watermarks thatare mapped to media identifying information.

In some examples, a monitoring entity (e.g., the Nielsen Company (US),LLC) can monitor media presentations to monitor exposure toadvertisements, monitor content consumption (e.g., exposure to atelevision show, exposure to a radio program, etc.), determineviewership/listenership demographics, etc. Such a monitoring entity canmonitor media signals to quickly and accurately identify media based onencoded watermarks in the signals. As such, monitoring entitiestypically prefer that watermarks are consistently encoded in mediasignals to enable accurate monitoring.

In some examples, watermarks are encoded by a media broadcaster prior tobroadcasting the media signals. In some examples, the watermarks can beencoded by a different entity in the media distribution process (e.g., acontent provider, a measurement entity, a local broadcaster, a cablebroadcaster, an online distributor, etc.). Due to the variety ofdifferent locations at which watermarks can be encoded, and the varietyof different entities that can encode watermarks, it can be difficult toensure reliable and consistent watermark encoding. One method ofimproving the overall reliability is to implement alarms that indicateto the operator when watermarks are not being encoded. Conventionally,an alarm setting is implemented at encoding locations (e.g., at a mediabroadcaster, etc.) to initiate an alarm and thereby alert an operatorthat the media is not being encoded with watermarks. However, mediasignals can include portions, such as commercials, that areintentionally not encoded with watermarks. Hence, such conventionaltechniques can trigger unwanted false alarms, which can lead to a “crywolf” effect, where alarms are no longer trusted as indicating an actualwatermark encoding issue that requires attention. While some alarms areconfigured with customizable tolerances, which enable a user to set aminimum outage duration to trigger an alarm, the durations of watermarkoutages can vary, which reduces the effectiveness of a selectedtolerance. A technique is needed that can identify whether a watermarkoutage is expected (e.g., due to a commercial break, due to programmingthat is intentionally not encoded with watermarks, etc.) or a watermarkoutage is unexpected (e.g., due to an error in watermark encoding, dueto an overall error in media signal transmission, etc.). As used herein,the term watermark outage refers to a portion of a media signal that isnot encoded with watermarks.

Example methods, apparatus, systems and articles of manufacture (e.g.,physical storage media) to dynamically analyze media signals andaccurately trigger watermark encoding outage alarms for unexpected(e.g., unintentional) watermark outages are disclosed herein. Someexample watermark outage monitoring techniques disclosed herein includemodeling the presence of watermarks in media signals to determinepatterns associated with watermark encoding. In some examples, onsettimes of watermark outages and/or durations of watermark outages can bemodeled and utilized to identify expected and/or unexpected watermarkoutages. For example, during an hour-long television broadcast, theremay be several expected watermark outages that occur at regularintervals (e.g., commercial breaks, infomercials, public serviceannouncements, etc.) with regular durations (e.g., two-minute commercialbreak, one-minute public service announcement, etc.). It would beundesirable for these expected watermark outages to trigger alarms. Bymodeling an onset time of these watermark outages, as well as a durationof these watermark outages, disclosed example watermark outagemonitoring systems can be implemented to trigger alarms when anunexpected watermark outage occurs (e.g., due to an error in thewatermark encoding process, a failure of the watermark encoding system,etc.), but not trigger an alarm when an expected watermark outage occurs(e.g., during times when commercials and other non-watermarked media istypically broadcast). An onset time can be, for example, a discretevalue associated with a time at which a watermark outage occurs (e.g.,3:32 PM, 16:32, etc.) or an elapsed time associated with an amount oftime that has passed since the last watermark outage (e.g., 33 minutessince the last watermark outage). For example, if commercial breakswithout watermark encoding typically happen at fifteen-minute intervalswithin a program (e.g., fifteen minutes past the hour, thirty minutespast the hour, forty-five minutes past the hour, etc.), and a watermarkoutage is detected at eight minutes past the hour, a disclosed examplewatermark outage monitor can determine that the detected watermarkoutage does not match an expected watermark outage represented in themodel and thereafter issue a watermark outage alert.

Some disclosed example watermark outage monitoring techniques determinewhether a detected watermark outage is an expected or unexpectedwatermark outage by analyzing a previous watermark outage onset time todetermine if a detected watermark outage represents an expectedtransition between watermark outages. In some examples, if the onsettime represents an elapsed time since the watermark outage, the onsettime itself may be utilized to determine if a detected watermark outagerepresents an expected transition between watermark outages (e.g.,without referencing a previous watermark outage onset time). Forexample, one type of television program may have a series of threeexpected watermark outages. The first outage may occur at ten minutespast the hour, which is expected to be followed by the second watermarkoutage at thirty minutes past the hour, which is finally expected to befollowed by the third watermark encoding outage at fifty minutes pastthe hour. Another type of television program can have a series of threeexpected watermark outages, with the first occurring at ten minutes pastthe hour, followed by the second watermark outage at twenty-five minutespast the hour, followed by the third expected watermark encoding outageat fifty minutes past the hour. In this example, if an outage isdetected at thirty minutes past the hour, and the previous outage wasdetected at twenty-five minutes past the hour, this may be an unexpectedwatermark outage, as this transition (from an outage at twenty-fiveminutes to an outage at thirty minutes) does not correspond with themodel's expected watermark outage state transitions.

In some disclosed examples, a duration of a detected watermark outagecan be compared to the model to determine if the detected watermarkoutage is an expected watermark outage. For example, if a watermarkoutage continues for seven minutes, when previously seen watermarkoutages represented by the model have durations of four or five minutes,the watermark outage monitor can decide to issue an alarm.

In some examples, a Markov model is utilized to analyze watermarkencoding data. Specifically, the Markov model can analyze the onset ofthe watermark outages (e.g., the times at which the watermark outagesbegin, etc.). For example, a Markov model can be continually trained andupdated based on watermark encoding data at a broadcaster. The Markovmodel can represent probabilities of a watermark outage occurring at anytime in the watermark encoding data based on a previous watermark outageor an elapsed time in a media signal (e.g., fifteen minutes into atelevision program, etc.). In some examples, the Markov model isestablished based on an initial training period, with an occurrence of anew state (e.g., a watermark outage that does not correspond to anexisting state in the model, etc.) resulting in a watermark outagealarm. The states of the Markov model can be based on onset timesassociated with the watermark outages. New states can be incorporated asexpected outage states into an updated model when they recur with afrequency satisfying (e.g., meeting or exceeding) a threshold,indicating that the new state may be a new expected watermark outage(e.g., a new commercial onset time, etc.).

In addition to states, state transitions (e.g., corresponding tosequential pairs of watermark outages represented in the model) can beanalyzed and included in the model. State transitions can be associatedwith probabilities of an occurrence of the transition. For example, theprobability of a state transition may increase in the model if moreoccurrences of that state transition are observed. In some examples, aninitial outlier watermark outage associated with a new state transitioncan, over time, become an expected watermark outage if the same statetransition occurs enough times such that the probability of the statetransition satisfies an occurrence threshold. Similarly, when a newstate (e.g., a new onset time for a watermark outage) is observed, itmay initially be an outlier watermark outage that triggers an alarm. Asused herein, an outlier watermark outage refers to an outage withcharacteristics that are not expected by the model (e.g., an unexpectedonset time, an unexpected transition between watermark outages, anunexpected duration, etc.), and may indicate a watermark outage that isunexpected (e.g., unintentional). However, if the same state occursoften enough to satisfy an occurrence threshold, it may be representedin the model as an expected outage state thereafter.

In some examples, clustering techniques (e.g., k-means clustering usinga gap statistic method to determine the number of clusters) can be usedto establish a second model, or a second transition state of theexisting model, based on watermark outage durations and/or watermarkoutage onset times (e.g., elapsed times since previous watermark outagesor discrete times). Such techniques are leveraged to take into accountthe observation that intentional watermark outages typically haverelatively consistent (e.g., but not exactly equal) durations with minorvariations (e.g., a due to lack of reception of the tuner from low poweroperation during nighttime of an AM station, etc.). Watermark encodingdata can be utilized to train the model to recognize clusters ofwatermark outage durations, and subsequently issue outage alarms when awatermark outage occurs that has a duration and/or an onset time, or acombination of a duration and onset time, that do not fall into one ofthe clusters as defined by sampled cluster and gap sizes. In someexamples, once a new watermark outage duration occurs with a frequencybeyond an occurrence threshold, indicating that the watermark outageshaving the new duration may not be unintentional watermark outages(e.g., outlier watermark outages) but rather intentional watermarkoutages, the model can be updated accordingly.

In some examples, additional or alternative modeling techniques can beimplemented to characterize watermark outages. For example, additionalproperties of a watermark outage (e.g., audio characteristics, such assilence and/or presence of a wrong code during the outage), can beutilized to enhance the accuracy and granularity of the model, forexample, by filtering them out or defining a state transition of themodel in terms of multiple properties.

These and other techniques, methods, apparatus, systems and articles ofmanufacture to dynamically analyze media signals for watermark outageanalysis and to issue accurate watermark outage alarms are disclosed ingreater detail below.

FIG. 1 is a block diagram of an example environment 100 for monitoringwatermarked media signals in accordance with the teachings of thisdisclosure. The example environment 100 includes an example media signal102, an example media broadcaster 104, an example watermark encoder 106,an example watermarked media signal 108, an example media transmitter110, an example broadcast media signal 112, an example watermark outagemonitor 114, an example watermark outage alert 116, an examplebroadcaster control center 118, and an example audience measuremententity 120.

The example media signal 102 of the illustrated example of FIG. 1 is asignal conveying media that is intended for broadcast to an audience.The media signal 102 is received by the media broadcaster 102. The mediasignal 102 can be an audio signal conveying media (e.g., a radiobroadcast, a podcast, etc.), an audiovisual signal conveying media(e.g., a television show, a movie, a commercial, etc.), or any othersignal conveying media. In some examples, the media signal 102 istransmitted to the media broadcaster 104 by a content creator, a contentdistributor, or another entity. Additionally or alternatively, the mediabroadcaster 104 may generate the media signal itself. The media signal102 can be conveyed wirelessly (e.g., via a network, etc.) or via adirect physical connection (e.g., cable, etc.).

The example media broadcaster 104 of the illustrated example of FIG. 1is an entity for transmitting media signals to a broad audience. Forexample, the media broadcaster 104 can receive a plurality of differentmedia signals conveying media and utilize transmission technology (e.g.,via antennas, via satellites, via cable, via the internet, etc.) to makethe media signals available to a large audience.

The example watermark encoder 106 of the illustrated example of FIG. 1encodes (e.g., embeds, inputs, etc.) watermarks into the media signal102 for subsequent identification of media conveyed by the media signal102. In some examples, the watermark encoder 106 embeds the watermarksin audio and/or video signal characteristics into the media signal 102such that the watermarks are sufficiently hidden (e.g., inaudible,invisible, etc.) to the audience, but still detectable and identifiableby the audience measurement entity 120. In some examples, the watermarksinclude information such as a program identifier, a station identifier,a timestamp, and/or any other information that can be useful inidentifying the media conveyed in the media signal 102. In someexamples, the watermark encoder 106 can selectively encode media signalsbased on whether identification information is available for the mediasignals. For example, a radio broadcast may have watermarks encodedthroughout a radio program being broadcast, but the commercial breaks inbetween the radio program may not have watermarks. Therefore, someportions of the media signal 102 may be intentionally transmittedwithout watermarks encoded into the media signal 102. While some mediasignals or portions of media signals may be intentionally transmittedwithout watermarks encoded into the signals, others may beunintentionally transmitted without watermarks in the media signals. Forexample, the watermark encoder 106 can encounter issues while encodingwatermarks that result in erroneous (e.g., ineffective) watermarks beingencoded into the signals, or no watermarks being encoded into thesignals at all. In some examples, the example broadcaster control center118 can control the watermark encoder 106 and take actions to initiatewatermark encoding, terminate watermark encoding, correct watermarkencoding issues, or otherwise interact with the watermark encodingprocess.

The example watermarked media signal 108 of the illustrated example ofFIG. 1 is the media signal output by the watermark encoder 106. Thewatermarked media signal 108 is similar to the media signal 102, butalso includes audio and/or video characteristics that have been modifiedto represent an identifying watermark. In some examples, the mediasignal 102 and the watermarked media signal 108 may be perceived by anaudience as being substantially identical, due to the encoding of thewatermarks in the watermarked media signal 108 in a manner that makesthe watermarks imperceptible to human senses. In some examples, portionsof the watermarked media signal 108 can be intentionally orunintentionally not encoded with watermarks.

The example media transmitter 110 transmits the watermarked media signal108 to an audience. The watermarked media signal 108, after having beenprocessed and transmitted by the media transmitter 110, can be referredto as the broadcast media signal 112. The example media transmitter 110can be any transmission technology for media signal transmission. Forexample, the media transmitter 110 can be one or more antennas, one ormore satellites, one or more network-based transmission systems, or oneor more cable systems, etc.

The example broadcast media signal 112 of the illustrated example ofFIG. 1 is the watermarked media signal 108 following transmission by themedia transmitter 110. The broadcast media signal 112 can be received byan audience (e.g., via an antenna, via a satellite dish, via a cableconfiguration, via a network, etc.).

The example watermark outage monitor 114 of the illustrated example ofFIG. 1 monitors the watermarked media signal 108 output by the watermarkencoder 106 to determine if the watermarked media signal 108 containswatermarks. The watermark outage monitor 114 is structured to generatewatermark encoding outage alerts when the watermarked media signal 108does not include encoded watermarks and the watermark outage monitor 114determines that such a watermark outage is unintentional. For example,the watermark outage monitor 114 can determine whether an encodingoutage appears to be intentional (e.g., due to a commercial break, dueto a transition between programming, etc.) or unintentional (e.g., dueto an error in the encoding process, due to a failure of the watermarkencoder 106, etc.). To make this determination, the watermark outagemonitor 114 of the illustrated example generates and utilizes modelsbased on the presence of watermarks in observed media signals. In someexamples, the models include information on onset times of watermarkoutages, durations of watermark outages, and/or other factors that canbe useful in analyzing patterns of watermark outages.

In some examples, the watermark outage monitor 114 continually monitorsthe watermarked media signal 108 as it is encoded by the watermarkedencoder 106. In some examples, the watermark outage monitor 114 monitorsthe broadcast media signal 112 in addition to, or alternatively to, thewatermarked media signal 108. The example watermark outage monitor 114outputs watermark outage alerts 116, which can be transmitted to thebroadcast control center 118, to alert the media broadcaster 104 of anissue with the watermark encoder 106. The watermark outage monitor 114can additionally or alternatively output the watermark outage alert 116to the audience measurement entity 120. In some examples, the watermarkoutage monitor 114 can transmit other information about watermarkoutages, such as any expected watermark outages (e.g., that did nottrigger alarms) to the audience measurement entity 120 and/or to thebroadcaster control center 118 for further analysis.

The example watermark outage alert 116 of the illustrated example ofFIG. 1 is an indicator of a watermark outage that is not expected by themodel(s) of the watermark outage monitor 114. The example watermarkoutage alert 116 can be any form of alert (e.g., an alarm, an actiontrigger, an email, a text message etc.) to indicate to the mediabroadcaster 104 and/or to the audience measurement entity 120 that thewatermark encoder 106 may not be working correctly. In some examples,the watermark outage alert 116 can additionally or alternatively be asignal or command to reset the watermark encoder 106 or otherwise resetthe operation of the watermark encoder 106 to a known operating state.In some examples, the watermark outage alert 116 may indicate an onsettime (e.g., a start time or an elapsed duration since a previouswatermark outage) of the unexpected watermark outage and/or a durationof the unexpected watermark outage. The watermark outage alert 116 canalso include supplementary information, such as whether there wassilence or prior encoding observed in the media signal, or any otherinformation that can be useful in diagnosing the issue causing thewatermark outage.

The example broadcaster control center 118 of the illustrated example ofFIG. 1 is an element of the media broadcaster 104 that has control overthe watermark encoder 106. The broadcast control center 118 receives thewatermark outage alert 116 to analyze the watermark outage and takecorrective action when necessary. For example, the broadcaster controlcenter 118 can, in response to receiving the watermark outage alert 116,troubleshoot the watermark encoder 106.

The example audience measurement entity 120 is an entity for measurementand monitoring of audiences associated with media programming and/oradvertisements (e.g., the Nielsen Company (US), LLC). The audiencemeasurement entity 120 can utilize watermarks to identify media andperform various tasks, such as audience size calculations, demographicscalculations, etc. The audience measurement entity 120 thereforedesires, and may rely upon, reliable watermark encoding. When theaudience measurement entity 120 receives a watermark outage alert 116,the audience measurement entity 120 can take corrective action tocorrect any data based on the media signals lacking watermarks, and/orto work with the media broadcaster 104 to address the problem. Theaudience measurement entity 120 may also receive additional informationfrom the watermark outage monitor 114 to ensure the monitoring processis working correctly and access models associated with expectedwatermark outages (e.g., to ensure commercials are being inserted atappropriate times and with appropriate durations). In some examples, thewatermark outage monitor 114 can be implemented partially or entirely atthe audience measurement entity 120.

In operation, the media signal 102 is received by the watermark encoder106. The watermark encoder 106 then encodes watermarks into the mediasignal 102 and outputs the watermarked media signal 108, which isreceived by the media transmitter 110 and transmitted as the broadcastmedia signal 112. The watermarked media signal 108 is additionallyreceived by the watermark outage monitor 114 to ensure the watermarkedmedia signal 108 includes watermarks. In the event that the watermarkoutage monitor 114 detects an unexpected watermark outage (e.g., due toan error with the watermark encoder 106, etc.), the watermark outagemonitor 114 issues the watermark outage alert 116. The watermark outagealert 116 is received by the broadcast control center 118 and isadditionally received by the audience measurement entity 120.

FIG. 2 is a block diagram of an example implementation of the watermarkoutage monitor 114 for detecting watermark outages constructed inaccordance with the teachings of this disclosure. The watermark outagemonitor 114 includes an example media receiver 202, an example watermarkdetector 204, an example outage analyzer 206, an example sequenceanalyzer 208, an example sequence model manager 210, an example sequencemodel evaluator 212, an example grouping analyzer 214, an examplegrouping model manager 216, an example grouping model evaluator 218, andan example alert generator 220.

The example media receiver 202 of the illustrated example of FIG. 2receives the watermarked media signal 108 and any other media signalsfor monitoring. The media receiver 202 is in communication with thewatermark encoder 106 and, thus, can have a physical connection with anoutput of the watermark encoder 106 of the media broadcaster 104 of FIG.1, a wireless connection, or any other connectivity to receive thewatermarked media signal 108 and/or the broadcast media signal 112.

The example watermark detector 204 of the illustrated example of FIG. 2determines whether the watermarked media signal 108 includes watermarks.In some examples, the watermark detector 204 continually monitors mediasignals received by the media receiver 202 to determine whetherwatermarks are present in the media signals. In some examples, thewatermark detector 204 can be implemented as a watermark decoder,capable of identifying watermarks and determining the informationincluded in the watermarks (e.g., program identification information,station identification information, timestamps, etc.). The watermarkdetector 204 provides indications of watermark outages to the outageanalyzer 206, which can use the watermark outage data to generatewatermark outage models, update watermark outage models, and/or comparea detected watermark outage to the watermark outage models. In someexamples, the watermark detector 204 provides watermark outage data(e.g., indications of detected watermarks) to the alert generator 224,which can, in response to receiving a detected watermark outage, accessinformation from the outage analyzer 206 to determine whether togenerate a watermark encoding outage alert 116.

The watermark detector 204 can provide watermark outage data in anyformat. For example, the watermark detector 204 can represent the stateof watermark encoding in the watermarked media signal 108 as a series ofbinary values sampled at a specified rate (e.g., one sample per second,one sample per minute, etc.). The watermark detector 204 can thereforegenerate an alarm sequence that can be analyzed by the outage analyzer206 to determine onset times associated with watermark outages anddurations associated with watermark outages. If an alarm sequence isgenerated with a zero value representing watermark encoding beingpresent (e.g., no watermark outage) and a one value representing awatermark outage, an example alarm sequence might be: 00011000111100. Ifthe start of this sequence represents a start time of zero seconds, andthe sample rate is one second, this alarm sequence indicates that afirst watermark outage occurred at four seconds, which had a duration oftwo seconds, and a second watermark outage occurred at nine seconds,which had a duration of four seconds.

The example outage analyzer 206 of the illustrated example of FIG. 2analyzes watermark outage data to generate models for watermark outages,update models for watermark outages, and evaluate models for watermarkoutages. In some examples, the outage analyzer 206 is capable ofgenerating and updating models directed to onset times of watermarkoutages and/or generating and updating models directed to durations ofwatermark outages. The outage analyzer 206 generates and updates modelsbased on the watermark outages detected by the watermark detector 204and evaluates detected watermark outages with respect to the model(s) toenable the alert generator 224 to generate alarms when an outlierwatermark outage occurs. In some examples, the outage analyzer 206evaluates an onset time and/or a duration of a detected watermark outageto determine whether the detected watermark outage corresponds to anexpected watermark outage represented in the model(s). In some examples,the outage analyzer 206 maintains a sliding time window that defines atime period for which watermark outage data is utilized in the sequencemodel and/or the grouping model. For example, the outage analyzer 206can be configured to consider watermark outage data as part of the modelfor a configured time period (e.g., for a month, for a year, etc.). Whendata is older than this specified time period and is no longer in thesliding time window, the outage analyzer 206 can update the sequencemodel and/or the grouping models accordingly, to no longer representthis data in the model(s). In some examples, only outlier data isremoved from the model(s) after it is outside the sliding time window.

The example sequence analyzer 208 of the illustrated example of FIG. 2generates, updates and evaluates an outage sequence model based on onsettimes (e.g., start times or elapsed times since previous watermarkoutages) associated with detected watermark outages. In some examples,the sequence analyzer 208 utilizes a Markov model (e.g., by generating adiscrete Markov chain) having states representing onset times associatedwith observed watermark outages. The sequence analyzer 208 cancontinually update the sequence model to represent data within thesliding time window maintained by the outage analyzer 208. The sequenceanalyzer 208 can analyze a detected watermark outage from the watermarkdetector 204 to evaluate the watermark outage against the sequence modeland determine whether the watermark outage corresponds to an expected(e.g., intentional) watermark outage. For example, if a watermark outagetypically occurs twenty-five minutes past the hour, as determined fromprior watermark outages represented as states in the sequence model, thesequence analyzer 208 can determine that a new detected watermark outageoccurring at twenty-five minutes past the hour is an expected watermarkoutage because the detected watermark outage corresponds to a state inthe model. The example sequence analyzer 208 may additionally oralternatively consider a previous watermark outage onset time as wellwhen analyzing a detected watermark outage, to determine whether atransition associated with the detected watermark outage is expected(e.g., the previous watermark outage onset time and the new detectedwatermark outage onset time correspond to two states in the modelconnected by a transition in the model). In examples where the sequenceanalyzer 208 utilizes a Markov model, the model stores transitionprobabilities, in addition to storing repeated sequences for confidencemetrics. In some examples, variations in the pattern for onset times mayresult in evaluating the duration of the outage as the outage progressesto determine whether or not to raise the alarm.

In some examples involving missing information (e.g., missing historicalwatermark outage data) due to communication failures, a temporary changein programming that impacts the occurrence of alarms, a higher order(sequence-dependent) probability dependency to the observed outages,etc., hidden node analysis is performed to determine if an observedelapsed time is 1) a combination of other elapsed times as part of aMarkov cycle (within cluster resolution), or 2) a sequence dependency tothe probability distribution (to account for higher order Markovbehavior). An example hidden node analysis based on a combination ofother elapsed times as part of a Markov cycle is described inassociation with FIGS. 7A-7B.

The example sequence model manager 210 of the illustrated example ofFIG. 2 generates and updates the sequence model. In some examples, thesequence model manager 210 can generate the model based on an initialwatermark outage dataset that has been previously collected. In someexamples, such an initial watermark outage dataset may be a learningdataset intended to represent typical watermark outage data from asimilar broadcast and/or broadcaster. In some examples, the sequencemodel manager 210 continually updates the model based on the watermarkoutages detected by the watermark detector 204. In some examples, thesequence model manager 210 updates the model at regular intervals (e.g.,after a specified time period, after a specified number of watermarkoutages, etc.) or in response to an update request (e.g., by thebroadcaster control center 118 of FIG. 1, by the audience measuremententity 120 of FIG. 1, etc.). The sequence model manager 210 may also beconfigured by the client to suppress alarms based on specific criteria,such as, but not limited to, based on elapsed time only, duration only,a combination of elapsed time and duration, or a confidence metric basedon 1) the percentage of noise (considered as outliers) in the trainingdata, 2) the presence of specific properties such as silence or priorencoding, or 3) the match along a prior observed sequence, etc.

To generate the sequence model, the sequence model manager 210 generatesstates associated with elapsed times for observed watermark outages andgenerates state transitions in the model. When a new state is initiallyobserved (e.g., a watermark outage has an onset time that has not beenpreviously observed), a state corresponding to this new onset time isinitially recognized as an outlier state. In some examples, anoccurrence threshold can be utilized to determine if an outlier statecan be elevated to an expected outage state (e.g., a state associatedwith an expected watermark outage onset time, such as a time typicallyassociated with a commercial break, etc.) in response to a state beingobserved numerous times. In addition to states, state transitions (e.g.,a transition from a first watermark outage with a first onset time to asecond watermark outage with a second onset time) are represented in themodel. When a detected watermark outage is being added to the model, thesequence model manager 210 can additionally access data pertaining tothe previous watermark outage to analyze the state transition that hasoccurred. If the sequence model manager 210 identifies a statetransition in the model that corresponds to the state transition of thedetected watermark outage currently being added to the model, then theprobabilities associated with the existing state transition are updatedbased on the newly added watermark outage. If the sequence model manager210 does not identify a state transition corresponding to the statetransition of the detected watermark outage currently being added to themodel, then a new state transition is added. Given the probability ofoccurrence of the new state transition is low, it will be treated as anoutlier. If the same transition occurs several more times and satisfiesan occurrence threshold, the sequence model manager 210 can elevate thestate transition to an expected outage state transition. In someexamples, an outlier state transition is elevated to be an expectedoutage state transition in response to a probability associated with thethreshold satisfying a configurable occurrence probability threshold. Anexample state and state transition diagram associated with a sequencemodel is depicted in FIG. 5F. In some examples, the sequence modelmanager 210 can additionally or alternatively generate states and assignwatermark outages to states based on other factors associated withwatermark outages. For example, states could include a propertyaccounting for whether the outage occurred during day-time programmingor night-time programming, to add confidence to the determination theoutage is due to low power transmission.

In some examples, to avoid excessive and inaccurate outliers, aclustering algorithm is additionally implemented by the sequence modelmanager 210 that determines separations between values (elapsed times ordurations) based on the creation of seed clusters from nearest neighborconsiderations or k-means clustering.

In some examples, the sequence model manager 210 continually updatesprobabilities of states and/or state transitions and the statuses ofstates and/or state transitions (e.g., as outlier or expected) based onthe data included in the sliding time window. In this manner, thesequence model manager 210 updates the model to represent more recentdata as opposed to older data. In this manner, outlier states and/oroutlier state transitions are removed from the model after a specifiedperiod, once occurrences of these outlier states and/or outlier statetransitions are outside of the sliding time window.

The example sequence model evaluator 212 of the illustrated example ofFIG. 2 evaluates detected watermark outages from the watermark detector204 against the sequence model. In some examples, the sequence modelevaluator 212 evaluates the detected watermark outage against the latestupdated model maintained by the sequence model manager 210. The samewatermark outage may be subsequently accounted for in the sequence modelwhen the sequence model manager 210 updates the model. In some examples,updating the model occurs simultaneously to evaluating the detectedwatermark outage against the sequence model. To evaluate the detectedwatermark outage against the sequence model, the sequence modelevaluator 212 determines an onset time for the detected watermark outageand accesses an onset time for the previous detected watermark outage.In some examples where the onset time is an elapsed time since theprevious watermark outage, the sequence model evaluator 212 determinesan onset time for the detected watermark (e.g., without necessarilyaccessing an onset time for the previous watermark outage). The sequencemodel evaluator 212 then determines whether the onset time associatedwith the detected watermark outage corresponds to an expected outagestate in the sequence model. The sequence model evaluator 212additionally or alternatively determines whether the state transitionbetween the previous state and the new state is associated with anexpected outage state transition in the sequence model. In someexamples, the sequence model evaluator 212 provides a determination tothe alert generator 220 as to whether the detected watermark outage isassociated with an expected outage state and an expected outage statetransition in the sequence model. In some examples, the sequence modelevaluator 212 can determine if the state transition is associated withan expected transition, without separately determining if the currentstate is an expected outage state, since an expected outage statetransition occurs between expected outage states. In such examples, thesequence model evaluator 212 can indicate to the alert generator 220whether the state transition was an expected outage state transition,without indicating whether the specific state of the detected watermarkoutage was expected.

The example grouping analyzer 214 generates, updates and evaluatesgrouping models based on durations associated with watermark outages. Insome examples, the grouping analyzer 214 utilizes clustering techniques(e.g., k-means clustering, nearest neighbor seed-based clustering, etc.)to identify common durations and/or onset times for watermark outages.For example, commercial breaks are typically known to have relativelycommon durations (e.g., thirty seconds, two minutes, etc.) and to occurat relatively common intervals (e.g., every eight minutes, every tenminutes, etc.). Hence, watermark outages having similar durations andonset times (e.g., falling within the same cluster) are likelyassociated with an expected watermark outage. The grouping analyzer 214can determine whether a detected watermark outage corresponds to anexisting cluster and/or whether a detected watermark outage exceeds aduration alarm threshold and provide such information to the alertgenerator 220. In the illustrated example, the alert generator 220issues a watermark outage alert if the detected watermark outage has aduration not corresponding to an existing cluster.

The example grouping model manager 216 of the illustrated example ofFIG. 2 generates and updates a grouping model based on detectedwatermark outages. The example grouping model manager 216 can generateand/or update the grouping model in tandem with the sequence modelmanager 210 generating and/or updating the sequence model. In someexamples, the grouping model manager is generated based on an initiallearning data set including watermark outages representative of typicaloutages for a specific broadcaster. In some examples, the grouping modelmanager 216 can continually build and update the model as new watermarkoutages are detected by the watermark detector 204. In some examples,the grouping model manager 216 updates the grouping model at regularintervals (e.g., after a specified time period, after a specified numberof watermark outages, etc.) or in response to an update request (e.g.,by the broadcaster control center 118 of FIG. 1, by the audiencemeasurement entity 120 of FIG. 1, etc.).

To generate the grouping model, the grouping model manager 216 createsclusters for watermark outages that have durations and/or onset times,or combinations of durations and onset times that are not represented bycurrent clusters (e.g., a first duration that has been previouslyobserved but at a different onset time than the first duration hastypically been observed at, etc.) in the model. When a new watermarkoutage is selected for addition to the model, the grouping model manager216 determines whether the duration associated with the selectedwatermark outage corresponds to an existing cluster. In response to thegrouping model manager 216 determining that the duration does correspondto an existing cluster, the selected watermark outage can be associatedwith the cluster. Conversely, in response to the grouping model manager216 determining that the selected watermark outage does not correspondto an existing cluster, a new outlier cluster can be created.

During an initial phase of the grouping model, when insufficient dataexists to form reliable clusters, the grouping model manager 216 can beconfigured with an initial duration alarm threshold that is stored inthe model and represents a maximum duration of an expected watermarkoutage. This initial duration alarm threshold can then be updated as themodel is further developed and more watermark outage durations areincorporated into the grouping model. For example, the duration alarmthreshold can be continually updated to be a percentage or an amounthigher than the duration of the cluster with the longest duration. Insome examples, when the grouping model manager 216 has developed asufficiently deep model (e.g., including numerous different clustersrepresentative of a long observation period or at least one entiresliding time window), the duration alarm threshold can be eliminated,and the clusters themselves can be utilized to determine if a watermarkoutage duration is expected. In such examples, if a duration of awatermark outage does not correspond to one of the existing clusters, itcan be determined to be an outlier, and the indication can betransmitted to the alert generator 220 for a potential watermark outagealert. Over time, if the grouping model manager 216 encounters anadditional number of watermark outages with durations that correspond tothe outlier duration, the grouping model manager 216 can determine thatthis cluster satisfies an occurrence threshold sufficient to become anew expected cluster, representing an expected duration of watermarkoutages.

In some examples, the grouping model manager 216 can work in cooperationwith the sequence model manager 210 to determine clusters associatedwith states of the sequence model. In some examples, the grouping modeland the sequence model are aspects of a single model, with durationserving as an additional means of characterizing a state in a Markovmodel. In some examples, the grouping model manager 216 can generateduration alarm thresholds specific to states associated with thesequence model. For example, if a specific state of the sequence modelis also associated with watermark outages having duration values thatare similar, a duration alarm threshold can be determined for thisspecific state. In such examples, the grouping model can accuratelyidentify an unexpected watermark outage that may occur if a durationexceeds the typical duration at this state (e.g., a watermark outagedoes not end at the end of a commercial break due to an encodingproblem, etc.).

The grouping model manager 216 can also reorganize clusters inassociation with different states of the sequence model to search for analternative model that more accurately represents underlying patterns inthe data. For example, if the sequence model is based on an hourlysystem (e.g., at the end of each hour the onset time resets to “zero” inthe model, if the onset time is a discrete onset time) but the durationsof commercial breaks vary every other hour, there may be multipleclusters associated with each state (e.g., with each onset time). Insuch examples, the grouping model manager 216 can search for additionalstates that better represent the data. In some examples, these statesmay represent hidden states (e.g., as in a Hidden Markov Model), thataccount for underlying factors contributing to the existence ofdifferent states. Example grouping models that can be generated by thegrouping model manager 216 are depicted in FIGS. 6A-6C. An examplereorganization process that can be undertaken by the grouping modelmanager 216 and the sequence model manager 210 is depicted in FIGS.7A-7B.

The grouping model evaluator 218 of the illustrated example of FIG. 2evaluates detected watermark outages from the watermark detector 204against the grouping model. In some examples, the grouping modelanalyzer 218 evaluates the detected watermark outage against the latestupdated model maintained by the grouping model manager 216. The samewatermark outage may be subsequently accounted for in the grouping modelwhen the grouping model manager 216 updates the grouping model. In someexamples, updating the model occurs simultaneously to evaluating thedetected watermark against the grouping model. To evaluate the detectedwatermark outage against the grouping model, the grouping modelevaluator 218 determines a duration and an onset time (e.g., an elapsedtime since a prior watermark outage) for the detected watermark outage.The grouping model evaluator 218 then determines if the durationassociated with the detected watermark outage and the onset timeassociated with the detected watermark outage are within a threshold ofan existing cluster in the grouping model. If the duration of thedetected watermark outage and the onset time of the detected watermarkoutage do fall within an existing cluster of the grouping model, thenthe grouping model evaluator 218 provides a determination to the alertgenerator 220 that the detected watermark outage has an expectedduration. Conversely, if the duration of the detected watermark outageand the onset time of the detected watermark outage do not fall withinan existing cluster of the grouping model, the grouping model evaluator218 provides a determination to the alert generator 220 that thedetected watermark outage does not correspond to an expected cluster,which may result in the alert generator 220 generating a watermarkoutage alert. In some examples, the grouping model evaluator 218 canadditionally or alternatively determine if the duration of the detectedwatermark outage exceeds a duration alarm threshold associated with thegrouping model and/or associated with one or more clusters of the stateof the detected watermark outage. In response to the duration of thedetected watermark outage exceeding the duration alarm threshold, thegrouping model evaluator 218 can provide a determination to the alertgenerator 220 to generate the watermark outage alert 116.

The alert generator 220 of the illustrated example of FIG. 2 generatesthe watermark outage alert 116 in response to a watermark outagedetected by the watermark detector 204 and a determination from theoutage analyzer 206 indicating that the watermark outage is unexpected.In some examples, the alert generator 220 generates the watermark outagealert 116 when (1) a watermark outage is detected and (2) either (a) thesequence model indicates that the detected watermark has an unexpectedwatermark outage onset time (e.g., an unexpected outage state and/orunexpected outage state transition) or (b) the grouping model indicatesthat the detected watermark has an unexpected duration, onset time orcombination of duration and onset time (e.g., not corresponding to acluster of the model) or an excessive duration (e.g., surpassing aduration alarm threshold). In some examples, the alert generator 220 canutilize any combination of these determinations to decide whether togenerate the watermark encoding outage alert.

In operation, the watermarked media signal 108 is received by the mediareceiver 202 and is additionally processed by the watermark detector 204to determine whether the watermarked media signal 108 includeswatermarks. The outage analyzer 206 processes the watermarked mediasignal 108 at the sequence analyzer 208 by generating and/or updating asequence model. The sequence model evaluator 212 can utilize thesequence model to evaluate watermark outages detected by the watermarkdetector 204 against the sequence model. The grouping analyzer 214processes the watermarked media signal 108 at the grouping model manager216 by generating and/or updating a grouping model. The grouping modelevaluator 218 can utilize the grouping model to evaluate watermarkoutages detected by the watermark detector 204 against the groupingmodel. In response to the watermark detector 204 detecting a watermarkoutage, and the sequence model evaluator 212 and/or the grouping modelevaluator 218 determining that the watermark outage is not an expectedwatermark outage, the alert generator 220 can generate the watermarkoutage alert 116.

While an example manner of implementing the watermark outage monitor 114of FIG. 1 is illustrated in FIG. 2, one or more of the elements,processes and/or devices illustrated in FIG. 2 may be combined, divided,re-arranged, omitted, eliminated and/or implemented in any other way.Further, the example media receiver 202, the example watermark detector204, the example outage analyzer 206, the example sequence analyzer 208,the example sequence model manager 210, the example sequence modelevaluator 212, the example grouping analyzer 214, the example groupingmodel manager 216, the example grouping model evaluator 218, the examplealert generator 220 and/or, more generally, the example watermark outagemonitor 114 of FIG. 1 may be implemented by hardware, software, firmwareand/or any combination of hardware, software and/or firmware. Thus, forexample, any of the example media receiver 202, the example watermarkdetector 204, the example outage analyzer 206, the example sequenceanalyzer 208, the example sequence model manager 210, the examplesequence model evaluator 212, the example grouping analyzer 214, theexample grouping model manager 216, the example grouping model evaluator218, the example alert generator 220 and/or, more generally, the examplewatermark outage monitor 114 could be implemented by one or more analogor digital circuit(s), logic circuits, programmable processor(s),programmable controller(s), graphics processing unit(s) (GPU(s)),digital signal processor(s) (DSP(s)), application specific integratedcircuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)). When reading any of theapparatus or system claims of this patent to cover a purely softwareand/or firmware implementation, at least one of the example mediareceiver 202, the example watermark detector 204, the example outageanalyzer 206, the example sequence analyzer 208, the example sequencemodel manager 210, the example sequence model evaluator 212, the examplegrouping analyzer 214, the example grouping model manager 216, theexample grouping model evaluator 218, the example alert generator 220is/are hereby expressly defined to include a non-transitory computerreadable storage device or storage disk such as a memory, a digitalversatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc.including the software and/or firmware. Further still, the examplewatermark outage monitor 114 of FIG. 1 may include one or more elements,processes and/or devices in addition to, or instead of, thoseillustrated in FIG. 2, and/or may include more than one of any or all ofthe illustrated elements, processes and devices. As used herein, thephrase “in communication,” including variations thereof, encompassesdirect communication and/or indirect communication through one or moreintermediary components, and does not require direct physical (e.g.,wired) communication and/or constant communication, but ratheradditionally includes selective communication at periodic intervals,scheduled intervals, aperiodic intervals, and/or one-time events.

Flowcharts representative of example hardware logic, machine readableinstructions, hardware implemented state machines, and/or anycombination thereof for implementing the watermark outage monitor 114 ofFIG. 2 is shown in FIGS. 3A, 3B, and 4. The machine readableinstructions may be an executable program or portion of an executableprogram for execution by a computer processor such as the processor 812shown in the example processor platform 800 discussed below inconnection with FIG. 8. The program may be embodied in software storedon a non-transitory computer readable storage medium such as a CD-ROM, afloppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associatedwith the processor 812, but the entire program and/or parts thereofcould alternatively be executed by a device other than the processor 812and/or embodied in firmware or dedicated hardware. Further, although theexample program is described with reference to the flowchartsillustrated in FIGS. 3A, 3B, and 4, many other methods of implementingthe example watermark outage monitor 114 may alternatively be used. Forexample, the order of execution of the blocks may be changed, and/orsome of the blocks described may be changed, eliminated, or combined.Additionally or alternatively, any or all of the blocks may beimplemented by one or more hardware circuits (e.g., discrete and/orintegrated analog and/or digital circuitry, an FPGA, an ASIC, acomparator, an operational-amplifier (op-amp), a logic circuit, etc.)structured to perform the corresponding operation without executingsoftware or firmware.

As mentioned above, the example processes of FIGS. 3A, 3B, and 4 may beimplemented using executable instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media.

“Including” and “comprising” (and all forms and tenses thereof) are usedherein to be open ended terms. Thus, whenever a claim employs any formof “include” or “comprise” (e.g., comprises, includes, comprising,including, having, etc.) as a preamble or within a claim recitation ofany kind, it is to be understood that additional elements, terms, etc.may be present without falling outside the scope of the correspondingclaim or recitation. As used herein, when the phrase “at least” is usedas the transition term in, for example, a preamble of a claim, it isopen-ended in the same manner as the term “comprising” and “including”are open ended. The term “and/or” when used, for example, in a form suchas A, B, and/or C refers to any combination or subset of A, B, C such as(1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) Bwith C, and (7) A with B and with C.

Example machine readable instructions 300 that may be executed by thewatermark outage monitor 114 to generate and/or update the sequencemodel and the grouping model are illustrated in FIGS. 3A-3B. Withreference to the preceding figures and associated descriptions, theexample machine readable instructions 300 of FIG. 3A begin with theexample watermark outage monitor 114 accessing watermark outage data(block 302). In some examples, the watermark outage data accessed by theoutage analyzer 206 indicates at least onset times and durationsassociated with watermark outages in a media signal. The watermarkoutage data can be in any format. For example, the watermark outage datacan be a series of binary values representing whether the media signalincludes watermark(s) or does not include watermark(s) during a sampleperiod. In some examples, the watermark detector 204 can provide thewatermark outage data to the outage analyzer 206.

At block 304, the example watermark outage monitor 114 updates a slidingtime window. In some examples, the outage analyzer 206 updates a slidingtime window by incrementing the sliding time window by a specifiedamount of time. For example, the sliding time window can be incrementedby a number of seconds, minutes, hours, days, etc. The sliding timewindow represents a duration of time for which watermark outages arerepresented by the sequence model and/or the grouping model. Forexample, the sliding time window can be configured such that thesequence model and/or the grouping model are updated based on watermarkoutages that have occurred within the time range of the sliding timewindow (e.g., within the previous three days, within the previous week,within the previous month, etc.).

At block 306, the example watermark outage monitor 114 selects awatermark outage from the watermark outage data to evaluate for thesequence model. In some examples, the sequence model manager 210 selectsa watermark outage from the watermark outage data to evaluate for thesequence model by selecting the watermark outages in sequential orderfor addition to the sequence model. The sequence model manager 210 canselect watermark outages in any order when selecting watermark outagesto update the sequence model.

At block 308, the example watermark outage monitor 114 determineswhether an onset time for the selected watermark outage and a previouswatermark outage onset time correspond to an existing state transitionin the model. In some examples, the sequence model manager 210 accessesa previous watermark outage from the watermark outage data anddetermines whether an onset time associated with the previous watermarkoutage and an onset time associated with the selected watermark outagecorrespond to an existing state transition. For example, if the selectedwatermark outage has an onset time at five minutes, and the previouswatermark outage has an onset time at two minutes, the sequence modelmanager 210 determines whether there is a state transition in thesequence model that connects the states corresponding to these onsettimes. In some examples, the watermark outage monitor 114 determinesdirectly based on the onset time for the selected watermark outagewhether the outage corresponds to an existing state transition in themodel, if the onset time is an elapsed time since a previous watermarkoutage. In response to determining that the onset time for the selectedwatermark outage corresponds to an existing state transition in thesequence model, processing transfers to block 312. Conversely, inresponse to the onset time for the selected watermark outage notcorresponding to an existing state transition in the sequence model,processing transfers to block 310.

At block 310, the example watermark outage monitor 114 adds a stateand/or state transition to the sequence model as an outlier state and/orstate transition. In some examples, the sequence model manager 210 addsa state and/or state transition to the sequence model as an outlierstate and/or state transition. For example, if the selected watermarkoutage has an onset time that does not correspond to an existing statein the sequence model, the sequence model manager 210 creates a newstate associated with the onset time of the selected watermark outage.Additionally or alternatively, if the selected watermark outage and theprevious watermark outage do not correspond to an existing statetransition in the sequence model, the sequence model manager 210 createsa new state transition between states of the sequence model. When a newstate and/or new state transition is added to the sequence model, theyare initially labeled as outlier states and/or outlier statetransitions, as they have only been witnessed once within the slidingtime window. In some examples, state transitions are not labeled asoutlier state transitions or expected outage state transitions, butrather are evaluated based on a probability associated with the statetransition when evaluating a detected watermark outage against thesequence model.

At block 312, the example watermark outage monitor 114 updatesprobabilities associated with existing state transitions based on theselected watermark outage onset time. In some examples, the sequencemodel manager 210 updates probabilities associated with the existingstate transitions based on the selected watermark outage onset time,such as by increasing a probability associated with the state transitioncorresponding to the selected watermark outage due to its occurrence.The introduction of the selected watermark outage into the sequencemodel, in combination with the sliding window, can increase or decreaseprobabilities associated with any of the state transitions in the model.

At block 314, the example watermark outage monitor 114 determineswhether there are any additional watermark outages to add to thesequence model. In some examples, the outage analyzer 206 can determinewhether there are any additional watermark outages to add to thesequence model by determining whether all the watermark outagesrepresented in the watermark outage data have been incorporated into thesequence model (e.g., added to the model by updating counts andprobabilities associated with states and state transitions correspondingto the watermark outages, added to the model by creating a new outlierstate and/or outlier transition, etc.). In response to determining thatthere are additional watermark outages to be added to the sequencemodel, processing transfers to block 306 to select a new watermarkoutage. Conversely, in response to there being no additional watermarkoutages to be added to the sequence model, processing transfers to block316.

At block 316, the example watermark outage monitor 114 removes anyoutlier states and/or outlier state transitions that are based on datathat is now outside the sliding time window. In some examples, thesequence model manager 210 removes any outlier states and/or outlierstate transitions that are based on data that is now outside the slidingtime window. For example, if the sequence model includes an outlierstate that occurred twenty-one days ago, and the sliding time windowonly includes watermark outage data that is from the previous twentydays, the sequence model manager 210 can remove the outlier state. Insome examples, the sequence model manager 210 removes only outlierstates and/or outlier state transitions that have fallen outside of thesliding time window. In other examples, the sequence model manager canadditionally or alternatively remove non-outlier data from the modelonce it is outside of the sliding time window. In some examples, theoutlier states and/or outlier state transitions are not entirelydeleted, such as if the outlier state still represents some data that iswithin the sliding time window. For example, if an outlier state wasobserved twenty-one days ago, and again five days ago (e.g., two timesin total), and the sliding time window is set to include data from thelast twenty days, the outlier state would still exist, but with only oneassociated watermark outage, as opposed to two.

At block 318, the example watermark outage monitor 114 determines ifstates in the sequence model represent expected outage states or outlierstates based on the updated model. In some examples, the sequence modelmanager 210 updates the status (e.g., expected status or outlier status)based on the updated model, due to potential changes that may haveoccurred when the model was updated to account for new watermark outagesobserved. For example, if sufficient watermark outages are observed witha particular onset time to make an outlier state associated with theparticular onset time satisfy an occurrence threshold, the outlier statemay be updated to be an expected outage state.

At block 320, the example watermark outage monitor 114 determines ifstate transitions in the sequence model represent expected outage statetransitions or outlier state transitions based on updated probabilitiesassociated with the state transitions. In some examples, the sequencemodel manager 210 determines whether probabilities associated with statetransitions in the sequence model represent expected outage statetransitions or outlier state transitions based on updated probabilitiesassociated with the state transitions (e.g., as updated at block 312).For example, the sequence model manager 210 can determine whetherprobabilities associated with state transitions in the sequence modelsatisfy an occurrence probability threshold. In response to an outlierstate transition satisfying the occurrence probability threshold, it canbe updated to be an expected outage state transition.

The example machine readable instructions 300 continue in FIG. 3B. Withreference to the preceding figures and associated descriptions, theexample machine readable instructions continue with the examplewatermark outage monitor 114 selecting a watermark outage from thewatermark outage data to evaluate for the grouping model (Block 322). Insome examples, the grouping model manager 216 selects a watermark outagefrom the watermark outage data to evaluate for inclusion in the groupingmodel. When selecting watermark outages for addition to the groupingmodel, the grouping model manager 216 can select watermark outages inany order.

At block 324, the example watermark outage monitor 114 determineswhether the selected watermark outage falls into an existing cluster. Insome examples, the grouping model manager 216 determines and/or accessesa duration value and/or an onset value associated with the selectedwatermark outage and compares it to durations and/or onset valuesassociated with clusters in a Markov model. For example, the groupingmodel manager 216 can determine whether a duration associated with theselected watermark outage is within a threshold of an existing clusterin the grouping model. In response to the selected watermark outagefalling into an existing cluster, processing transfers to block 326.Conversely, in response to the selected watermark not falling into anexisting cluster, processing transfers to block 332.

At block 326, the example watermark outage monitor 114 adds the selectedwatermark outage to the cluster. In some examples, the grouping modelmanager 216 adds the selected watermark outage to the existing clusterthat the selected watermark falls into by storing the watermark outageduration and/or onset time in association with the cluster. In someexamples, the grouping model manager 216 stores a time when thewatermark outage occurred, which can be useful when determining if thewatermark outage remains within the sliding time window in the future.In some examples, the grouping model manager 216 stores an indicationthat an additional duration falling within a cluster has occurred (e.g.,by incrementing a count of watermark outages that have had a durationfalling within a cluster).

At block 328, the example watermark outage monitor 114 determineswhether there are any additional watermark outages to add to the model.In some examples, the grouping model manager 216 determines, based onthe watermark outage data (e.g., the watermark outage data accessed atblock 302) whether there are any watermark outages to add to thegrouping model (e.g., any watermark outages not currently represented bythe grouping model). In response to there being watermark outages to addto the grouping model, processing transfers to block 322 to select a newwatermark outage to evaluate for the grouping model. Conversely, inresponse to there not being watermark outages to add to the groupingmodel, processing transfers to block 330.

At block 330, the example watermark outage monitor 114 removes anyoutlier watermark outages that are outside the sliding time window fromthe grouping model. In some examples, the grouping model manager 216removes any outlier watermark outages from the grouping model that areno longer within the sliding time window. In some examples, the groupingmodel manager 216 can additionally delete non-outlier watermark outageswhen they have fallen outside of the sliding time window.

At block 332, the example watermark outage monitor 114 stores theselected watermark outage as an outlier. In some examples, the groupingmodel manager 216 stores the selected outage as an outlier, andadditionally stores the time that the watermark outage occurred inassociation with the outlier to enable future pruning when the outlieris no longer within the sliding time window (e.g., at block 330).

At block 334, the example watermark outage monitor 114 determineswhether a plurality of outliers exists within a threshold duration ofthe selected watermark outage that represent a new expected cluster. Insome examples, the grouping model manager 216 determines whether aplurality of outliers exist within a threshold duration and/or athreshold onset time of the selected outage that represent a newexpected cluster. For example, if a first outlier has a duration of tenminutes, and there are three outliers having durations within a minuteof the first outlier, the grouping model manager 216 can determinewhether (1) these four outliers are sufficiently similar with respect tooutage duration and/or onset time to form an expected cluster, and (2)whether the four outliers constitute a sufficient quantity of outliersto form an expected cluster. In some examples, the grouping modelmanager 216 can determine whether a new expected cluster exists at aregular interval (e.g., once a day, once a week, etc.), in response toan action (e.g., a user requesting that the grouping model be updated,etc.), or at any other time.

At block 336, the example watermark outage monitor 114 creates a newcluster. In some examples, the grouping model manager 216 creates a newcluster with the outliers which are within a threshold distance of eachother, and which satisfy a quantity of outliers required to form acluster.

Example machine readable instructions 400 that may be executed by theexample watermark outage monitor 114 to analyze a detected watermarkencoding outage and determine if a watermark outage alert should begenerated are illustrated in FIG. 4. With reference to the precedingfigures and associated descriptions, the example machine readableinstructions 400 begin with the example watermark outage monitor 114accessing a media signal (block 402). In some examples, the mediareceiver 202 accesses the watermarked media signal 108.

At block 404, the example watermark outage monitor 114 detects a newwatermark encoding outage in the media signal. In some examples, thewatermark detector 204 can detect a new watermark encoding outage in themedia signal as the media is being received by the media receiver 202.The example watermark detector 204 can perform live monitoring tocollect watermark outage data and initiate processes to issue watermarkoutage alerts when necessary.

At block 406, the example watermark outage monitor 114 determines anonset time for the new watermark encoding outage. In some examples, thewatermark detector 204 can determine an onset time for the new watermarkencoding outage by recording a timestamp associated with the time atwhich watermarks are no longer detected in the watermarked media signal108 (e.g., representing the start of an outage). In some examples, thewatermark outage monitor 114 determines an onset time for the newwatermark encoding outage by determining an elapsed time since aprevious watermark encoding outage.

At block 408, the example watermark outage monitor 114 accesses aprevious watermark encoding outage onset time. In some examples, thesequence model evaluator 212 accesses a previous watermark encodingoutage onset time by determining the previous watermark outage evaluatedby the sequence model evaluator 212 and retrieving the onset timeassociated with this previous watermark outage. In some examples wherethe onset time is an elapsed time since the previous watermark outage,block 408 can be bypassed, and processing transfers directly from block406 to block 410.

At block 410, the example watermark outage monitor 114 determineswhether the new watermark outage onset time and the previous watermarkoutage onset time correspond to a state transition in the sequencemodel. In some examples, the example sequence model evaluator 212determines whether the new watermark outage onset time (e.g., of thewatermark outage detected at block 404) and the previous watermarkoutage onset time correspond to a state transition in the sequencemodel. In some examples where the new watermark outage onset time is anelapsed time since the previous watermark outage, the watermark outagemonitor determines whether the new watermark outage onset timecorresponds to a state in the sequence model. In response to the newwatermark outage onset time and the previous watermark outage onset timecorresponding to a state transition in the sequence model, processingtransfers to block 412. Conversely, in response to the new watermarkoutage onset time and the previous watermark outage onset time notcorresponding to a state in the sequence model, processing transfers toblock 418.

At block 412, the example watermark outage monitor 114 determineswhether the state transition is an expected outage state transition. Insome examples, the sequence model evaluator 212 determines whether thestate transition is an expected outage state transition based on data(e.g., a label associated with the transition) stored in the sequencemodel. In some examples, the sequence model evaluator 212 can determinewhether the state transition is an expected outage state transitionbased on a probability associated with the state transition satisfyingan occurrence probability threshold. In response to the state transitionbeing an expected outage state transition, processing transfers to block414. Conversely, in response to the state transition not being anexpected outage state transition, processing transfers to block 418.

At block 414, the example watermark outage monitor 114 accesses aduration of the new watermark encoding outage. In some examples, thegrouping model evaluator 218 accesses a duration of the new watermarkencoding outage. In some examples, the duration may be determined by thewatermark detector 204.

At block 416, the example watermark outage monitor 114 determineswhether the new watermark encoding outage falls into an existing clusterof the outage grouping model. In some examples, the grouping modelevaluator 218 determines whether the duration and/or onset time of thenew watermark encoding outage falls into an existing cluster of theoutage grouping model by determining whether the duration and/or onsettime of the new watermark encoding outage is within a threshold range ofdurations and/or onset times corresponding to an existing cluster in theoutage grouping model.

At block 418, the example watermark outage monitor 114 generates awatermark encoding outage alert. In some examples, the alert generator220 generates a watermark encoding outage alert to indicate that anunexpected watermark outage has been observed. The watermark encodingoutage alert generated by the alert generator 220 can have any form(e.g., an email or other message, an entry in data log, an indication ona computing device, etc.). In some examples, the watermark outage alert116 can additionally or alternatively be a signal or command to resetthe watermark encoder 106 or otherwise reset the operation of thewatermark encoder 106 to a known operating state.

At block 420, the example watermark outage monitor 114 labels the newwatermark encoding outage as an expected watermark outage. In someexamples, the alert generator 220 labels the new watermark encodingoutage as an expected watermark outage. In some examples, the alertgenerator 220 takes no action to specifically label the new watermarkencoding outage as expected, but somehow indicates that a watermarkencoding outage alert does not need to be issued (e.g., labeling thewatermark outage as already-processed, deleting the watermark outagefrom the pending/processing queue, etc.).

FIGS. 5A-5E represent example watermark detection plots 500A, 500B,500C, 500D, 500E for a portion of an example data collection period. Forexample, the watermark detection plots 500A, 500B, 500C, 500D, 500E eachrepresent one hour of time in a data collection period includingmultiple hours of data, and represent watermark outages in one or moremedia signal(s) analyzed by the watermark outage monitor 114. The timeperiods represented by the example watermark detection plots 500A, 500B,500C, 500D, 500E represent different combinations of watermark outageonset times that may be observed in an example data collection period.The watermark monitor 114 can divide the data collection period into anyinterval when performing onset time analysis, or alternatively analyzethe entire data collection period as a whole, with a start time of zerominutes and an end time corresponding to a time at which the mediasignal(s) concluded. The watermark detection plots 500A, 500B, 500C,500D, 500E include time axes spanning horizontally across the plots,beginning at zero minutes on the left end and concluding at sixtyminutes on the right end. The vertical axes include a discreteindication of whether the media signal included watermarks at a giventime, or did not include watermarks at the given time.

A first watermark detection plot 500A of the illustrated example of FIG.5A indicates that watermarks were detected throughout the hour period(e.g., “Hour 1”), except for at a first ten-minute onset time watermarkoutage 502A. The first ten-minute onset time watermark outage 502A hasan onset time slightly before ten minutes, at approximately eightminutes. However, for simplicity, watermark outages with onset timesgenerally around ten minutes are referred to as ten-minute onset timewatermark outages. Similarly, all of the watermark outages representedin the watermark detection plots 500A, 500B, 500C, 500D, 500E, aredescribed with approximate onset times, and are rounded to a centralvalue associated with the plurality of watermark outage onset times(e.g., ten minutes, forty-five minutes, fifty-five minutes, etc.) forsimplicity. In some examples, the onset times associated with thewatermark outages could be defined as elapsed times since a previouswatermark outage, as opposed to discrete times. The first ten-minuteonset time watermark outage 502A has a duration of approximately twominutes.

A second watermark detection plot 500B of the illustrated example ofFIG. 5B indicates that there was a second ten-minute onset timewatermark outage 502B detected during this time period (e.g., “Hour 2”),as well as a first forty-five-minute onset time watermark outage 504Band a first fifty-five-minute watermark outage 506B detected during thistime period. In examples where watermark outage onset times are elapsedtimes since previous watermark outages, the first forty-five-minuteonset time watermark outage 504 b has an onset time of thirty-fiveminutes. Similarly, the first fifty-five-minute watermark outage 506 bhas an onset time of ten minutes when onset time is determined based onelapsed times since the previous watermark outage. The duration of thesecond ten-minute onset time watermark outage 502B is slightly longerthan the duration of the first ten-minute onset time watermark outage502A.

A third watermark detection plot 500C of the illustrated example of FIG.5C indicates that there was a third ten-minute onset time watermarkoutage 502C detected during this time period (e.g., “Hour 3”), as wellas a forty-five-minute onset time watermark outage 504C detected duringthis time period. The third ten-minute onset time watermark outage 502Chas a shorter duration than the first and second ten-minute onset timewatermark outages 502A, 502B, and the second forty-five-minute onsettime watermark outage 504C has a longer duration than the firstforty-five-minute onset time watermark outage 504B. In some examples,watermark outage durations can have significant variance, which in somecases can limit the accuracy with which duration can be usedindependently to identify unexpected watermark outages. In suchexamples, a combined approach of analyzing watermark outage onset timesand watermark outage durations can be especially beneficial in issuingaccurate watermark outage alerts. Additionally, to account for variancein onset times of watermark outages, clustering can be utilized tocluster similar onset times (e.g., 502A, 502B, 502C, etc.).

A fourth watermark detection plot 500D is depicted in the illustratedexample of FIG. 5D. The fourth watermark detection plot 500D indicatesthat there was a fourth ten-minute onset time watermark outage 502Ddetected during this time period (e.g., “Hour 4”), a thirdforty-five-minute onset time watermark outage 504D, and a secondfifty-five-minute onset time watermark outage 506D detected during thetime period.

A fifth watermark detection plot 500E of the illustrated example of FIG.5E indicates that there was a fifth ten-minute onset time watermarkoutage 502E, as well as a fourth forty-five-minute onset time watermarkoutage 504E, a third fifty-five-minute onset time watermark outage 506Eand a first thirty-seven-minute onset time watermark outage 508Edetected during this time period (e.g., “Hour N”). The firstthirty-seven-minute onset time watermark outage 508E does not correspondto an onset time of any of the other watermark detection plots 500A,500B, 500C, 500D, 500E. This may indicate that the firstthirty-seven-minute onset time watermark outage 508E corresponds to anoutlier watermark outage, unless other portions of the example datacollection period include additional thirty-seven-minute onset timewatermark outages sufficient to satisfy an occurrence threshold.

FIG. 5F is a state diagram 500F based on the data collection period ofthe watermark detection plots of FIGS. 5A-5E. The state diagram 500Fincludes a plurality of states 502F, 504F, 506F, 508F, 510F and statetransitions 512F, 514F, 516F, 518F, 520F, 522F, 524F. In some examples,the state diagram 500F is representative of a Markov model.

The states 502F, 504F, 506F, 508F, 510F of the state diagram 500F areassociated with various onset times from the example data collectionperiod of the watermark detection plots of FIGS. 5A-5E. The first state502F corresponds to watermark outages with an onset time of ten minutes,such as the first ten-minute onset time watermark outage 502A, thesecond ten-minute onset time watermark outage 502B, etc. The secondstate 504F corresponds to watermark outages with an onset time offorty-five minutes, such as the first forty-five-minute onset timewatermark outage 504B, the second forty-five-minute onset time watermarkoutage 504C, etc. The third state 506F corresponds to watermark outageswith an onset time of fifty-five minutes, such as the firstfifty-five-minute onset time watermark outage 506B, the secondfifty-five-minute onset time watermark outage 506D, etc. The fourthstate 508F corresponds to watermark outages with an onset time ofthirty-seven minutes, such as the first thirty-seven minute-onset timewatermark outage 508E. In some examples, the states 502F, 504F, 506F,508F, 510F can be defined by onset times equal to elapsed times sinceprevious watermark outages. For example, if the states 502F, 504F, 506F,508F, 510F are defined by elapsed times since previous watermarkoutages, the fourth state 506F can be associated with an onset time often minutes, as the watermark outages at fifty-five minutes are precededby a watermark outage at forty-five minutes in the watermark outageplots of FIGS. 5A-5E, resulting in an onset time of ten minutes (e.g.,ten minutes preceding the fifty-five minute watermark outage since theprevious watermark outage) for the fourth state 506F. In some examples,any of these states 502F, 504F, 506F, 508F can be determined to beexpected outage states and/or outlier states, depending on whether thereare sufficient occurrences of watermark outages with onset timescorresponding to the respective states to satisfy an occurrencethreshold.

The state diagram 500F additionally includes a fifth state 510F that isassociated with discarded outlier states. For example, when dataassociated with an outlier state is no longer within the sliding timewindow, these outlier states may be “discarded.” In some examples, theseoutlier states are permanently deleted from the sequence model. In theexample state diagram 500F, the fifth state 510F remains in the model,and has an associated state transition 520F with a correspondingprobability of transition to the fifth state 510F. Upon updating themodel, this fifth state 510F can be deleted, and the corresponding statetransition 520F can be deleted as well, updating probabilitiesassociated with other state transitions in the process (e.g., updatingthe probabilities associated with state transitions 522F and 518F).

The state diagram 500F includes state transitions 512F, 514F, 516F,518F, 520F, 522F, 524F. The first transition 512F is associated withconsecutive watermark outages having onset times of ten minutes,resulting in the first state 502F occurring twice, consecutively. Forexample, if two consecutive hours with watermark outage datacorresponding to the first watermark detection plot 500A occurred, twoof the first ten-minute onset time watermark outages 502A would occurconsecutively, corresponding to the first transition 512F. The firststate transition 512F has a probability of 0.25, representing a twentyfive percent chance, according to the sequence model, of the statetransition occurring when at the first state 502F.

The second state transition 514F is associated with transitioning fromthe first state 502F to the second state 504F. For example, the secondwatermark detection plot 500B includes the second ten-minute onset timewatermark outage 502B followed by the first forty-five-minute onset timewatermark outage 504B, corresponding to the second state transition514F. The second state transition 514F has a probability of 0.72,representing a seventy two percent chance, according to the sequencemodel, of the second state transition 514F occurring when at the firststate 502F.

The third state transition 516F is associated with transitioning fromthe first state 502F to the fourth state 508F. For example, the fifthwatermark detection plot 500E includes the fifth ten-minute onset timewatermark outage 502E followed by the first thirty-seven-minute onsettime watermark outage 508E, corresponding to the third state transition516F. The third state transition 516F has a probability of 0.01,representing a one percent chance, according to the sequence model, ofthe third state transition 516F occurring when at the first state 502F.In some examples, this probability may not satisfy an occurrenceprobability threshold, resulting in the third state transition 516Fbeing an outlier state transition. In such examples, an occurrence ofthe third state transition 516F may result in the alert generator 220 ofthe watermark outage monitor 114 issuing a watermark outage alert 116.

The fourth state transition 518F is associated with transitioning fromthe second state 504F to the first state 502F. For example, if twoconsecutive hours with watermark outage data corresponding to the thirdwatermark detection plot 500C occurred, an instance of the secondforty-five-minute onset time watermark outage 504C would be followed byan instance of the third ten-minute onset time watermark outage 502C,corresponding to the fourth transition 518F. The fourth state transition518F has a probability of 0.2, representing a twenty percent chance,according to the sequence model, of the fourth state transition 518Foccurring when at the second state 504F.

The fifth state transition 520F is associated with transitioning fromthe second state 504F to the fifth state 510F, which representsdiscarded outlier states. The fifth state transition 520F has aprobability of 0.01, representing a one percent chance, according to thesequence model, of the fifth state transition 520F occurring when at thesecond state 504F.

The sixth state transition 522F is associated with transitioning fromthe second state 504F to the third state 506F. For example, in thesecond watermark detection plot 500B, the first forty-five-minute onsettime watermark outage 504B is followed by the first fifty-five-minuteonset time watermark outage 506B, corresponding to the sixth statetransition 522F. The sixth state transition 522F has a probability of0.79, representing a seventy-nine percent chance, according to thesequence model, of the sixth state transition 522F occurring when at thesecond state 504F.

The seventh state transition 524F is associated with transitioning fromthe third state 506F to the first state 502F. For example, if twoconsecutive hours with watermark outage data corresponding to the secondwatermark detection plot 500B occurred, an instance of the firstfifty-five-minute onset time watermark outage 506B would be followed byan instance of the second ten-minute onset time watermark outage 502B,corresponding to the seventh state transition 524F. The seventh statetransition 524F has a probability of 1.0, representing a one-hundredpercent chance, according to the sequence model, of the seventh statetransition 524F occurring when at the third state 506F. This one-hundredpercent chance indicates that in all of the data observed andincorporated into the model, an occurrence of the third state 506Fresulted in a transition to the first state 502F.

FIG. 6A includes an example first outage duration plot 600A includingtwo example outage onset times associated with similar durations,represented by two outage clusters. The outage duration plots 600A,600B, 600C of FIGS. 6A-6C include horizontal axes for plotting watermarkoutage durations, and vertical axes for representing discrete onsettimes of watermark outages. The vertical axis of the first outageduration plot 600A includes two onset times (e.g., associated with twostates in a Markov model): a first onset time 602 of ten minutes and asecond onset time 604 of forty-five minutes. For example, the firstonset time 602 can be associated with the first state 502F of the statediagram 500F of FIG. 5F. Similarly, the second onset time 604 can beassociated with the second state 504F of the state diagram 500F of FIG.5F. A plurality of watermark outage durations are plotted for the firstonset time 602 and the second onset time 604. For example, a firstwatermark outage duration 606 is associated with a watermark outage thathad an onset time of ten minutes (e.g., associated with the ten-minuteonset time state in the Markov model). Similarly, a second watermarkoutage duration 608 that is shorter than the first watermark outageduration 606 is associated with a watermark outage that had an onsettime of ten minutes. When a clustering algorithm is utilized to clusterthe watermark outage data of the first outage duration plot 600A, afirst cluster 610 includes all of the watermark outages in both thefirst onset time 602 and the second onset time 604, as represented bythe two dotted vertical lines representing the edges (e.g., limits) ofthe cluster. A second cluster 612 includes four watermark outages havingan onset time of forty-five minutes, and a duration approximately thesame as the duration of watermark outages associated with the firstcluster 612. The example clustering technique illustrated in FIGS. 6A-6Cand 7A-7B performs clustering based on onset times (e.g., elapsed timessince a previous outage or discrete times) and durations associated withwatermark outages. In some examples, clustering could be performed onduration alone, enabling clusters that include watermark outages with avariety of onset times.

In the example outage duration plot 600A, the data is highlypredictable, in that all of the watermark durations are very similar.When comparing newly detected watermark outages to a grouping modelconstructed for the data associated with the first outage duration plot600A, an outlier can be detected if a duration and onset timecombination of the watermark outage does not correspond to the firstcluster 610 or the second cluster 612.

FIG. 6B is an example second outage duration plot 600B including threeexample watermark outage clusters. In addition to the first onset time602 and the second onset time 604, the watermark outage data representedin the second outage duration plot 600B includes a third onset time 614of fifty-five minutes. The third onset time 614 can be associated withthe third state 506F of the state diagram 500F of FIG. 5F. The thirdonset time 614 includes a plurality of watermark outage durations havinga duration longer than the watermark outages of the first cluster 610and the second cluster 612. When a clustering algorithm is utilized tocluster the watermark outage duration data of the second outage durationplot 600B, a third cluster 616 includes the watermark outages having thethird onset time 614 and durations longer than the first cluster 610 andthe second cluster 612. In this example, despite there being additionalwatermark outages having a different onset time and duration associatedwith the third cluster 616, the data is very predictable in that each ofthe onset times 602, 604, 614 include watermark outages that haveduration values that are similar to one another. Hence, when thewatermark outage monitor 114 analyzes a detected watermark outageagainst the sequence model and the grouping model, it can clearlydetermine whether the detected watermark outage has an onset timecorresponding to the onset times 602, 604, 614 and then, determinewhether the watermark outage has a duration associated with one of theclusters 610, 612, 616. In some examples (e.g., as illustrated in FIGS.7A-7B), an onset time may be associated with more than one cluster dueto watermark outages having different durations but the same onset time.In such examples, it may be more challenging or time-consuming todetermine an outlier watermark outage, as an alarm cannot be triggeredwith confidence based on an onset time until a longest durationassociated with the clusters corresponding to the onset time has passed.A hidden node analysis can be performed in these less-predictablesituations to attempt to identify a more predictable clusteringsolution.

The watermark outage monitor can additionally or alternatively determinewhether the detected watermark outage has a duration falling into one ofthe clusters 610, 612, 616 associated with the onset times 602, 604,614. For example, in response to detecting a watermark outage with thethird onset time 614 that has a watermark outage duration correspondingto the first cluster 610 and the second cluster 612, the watermarkoutage monitor 114 may determine that an alert should be generated, asthe third onset time 614 is associated with the second cluster 616.

The second outage duration plot 600B additionally includes a durationalarm threshold 618 representative of an example threshold that can beimplemented to limit the maximum duration of an expected watermarkoutage duration. For example, the duration alarm threshold 618 isconfigured to a value slightly higher than the upper duration edge(e.g., limit) of the third cluster 616, such that if a watermark outagecontinues beyond the duration alarm threshold 618, the watermark outagemonitor 114 can issue an alarm, as no cluster has a duration exceedingthe duration alarm threshold 618. In some examples, a plurality ofduration alarm thresholds can be implemented that are associated withdifferent clusters. For example, if a watermark outage is detected thathas an onset time corresponding to the first onset time 602, based onthe first onset time 602 being associated only with the first cluster610, a duration alarm threshold could be utilized with a value that isslightly higher than the upper duration edge (e.g., limit) of the firstcluster 610.

FIG. 6C is an example third outage duration plot 600C depicting anoutlier duration 620 exceeding the example duration alarm threshold 618.The third outage duration plot 500C includes a watermark outageassociated with the third onset time 614. However, the watermark outagehas a duration exceeding the duration associated with the third cluster616, which is the only cluster associated with the third onset time 614.Therefore, the watermark monitor 114 identifies the detected watermarkoutage as an outlier watermark outage based on the watermark outagehaving a onset time and duration combination that does not correspond tothe any of the clusters 610, 612, 616. Additionally or alternatively,the watermark monitor 114 can identify the detected watermark outage asan outlier watermark outage based on the duration associated with theoutage exceeding the duration alarm threshold 618.

FIG. 7A is an example fourth outage duration plot 700A includingwatermark outages represented by a plurality of clusters. The fourthoutage duration plot 700A includes the first onset time 602, the secondonset time 604, and the third onset time 614. The watermark outage datarepresented by the fourth outage duration plot 700A is different thanthe data of the first, second, and third outage duration plots 600A,600B, 600C. In contrast to the watermark outage data of the first,second, and third outage duration plots 600A, 600B, 600C, the watermarkoutage data of the fourth outage duration plot 700A includes watermarkoutage durations falling into different clusters, but with the sameonset times. For example, the second onset time 604 includes somewatermark outage durations falling into a second cluster 704 and somewatermark outage durations falling into a third cluster 706. Similarly,the third onset time 614 includes some watermark outage durationsfalling into a fourth cluster 708 and some watermark outage durationsfalling into a fifth cluster 710. Conversely, for example, the firstonset time 602 has all watermark outage durations falling into a firstcluster 702. The watermark monitor 114 can determine whether a durationassociated with the first onset time 602 corresponds to an expectedduration by determining whether it falls into the first cluster 702, butit may have difficulty determining whether a duration associated withthe second onset time 604 or the third onset time 614 are expected dueto these onset times being associated with more than one cluster. Insome examples, the watermark monitor 114 can reorganize the sequencemodel and/or the grouping model to enable a more predictable and/or moreaccurate model.

FIG. 7B is an example outage duration plot 700B representing the outagedata of FIG. 7A reorganized in an adjusted model. The example adjustedmodel represents one possible solution to reorganizing the watermarkoutages by determining more specific onset times from a larger timeresolution that are associated with the clusters 704, 706, 708, 710. Themodel(s) can alternatively be adjusted or reorganized in any manner tointroduce more regularity (e.g., predictability) into the model toenable the watermark monitor 114 to accurately evaluate detectedwatermark outages against the model(s) and to set tighter tolerances foralarm duration thresholds and cluster edges. Specifically, clusterswhich are associated with a unique combination of onset time andduration can enable enhanced predictability and speed of analysis of adetected watermark outage.

The outage duration plot 700B includes the first onset time 602, withthe same watermark outage durations included in this first onset time602, since reorganization is not required for these watermark outages(e.g., all of the watermark outages in the data with this onset timehave durations corresponding to the same cluster). The outage durationplot 700B includes a fifth onset time 712 and a sixth onset time 714,corresponding to two separate onset times that help explain thedifferent durations represented in the second onset time 604. In thisexample, the fifth onset time 712 corresponds to an onset time offorty-five minutes occurring in a first hour, and the sixth onset time714 corresponds to an onset time of forty-five minutes occurring in asecond, consecutive, hour. Hence, the sixth onset time 714 canadditionally be referred to as being one-hundred-and-five minutes in atwo hour sample period. This could occur, for example, if a first hourof programming has a different advertisement scheduling configurationwhich has shorter advertisements, than a second hour of programming. Thewatermark outages of the fifth onset time 712 have durationscorresponding to the second cluster 704, and the watermark outages ofthe sixth onset time 714 have durations corresponding to the thirdcluster 706. While discrete times (e.g., forty-five minutes into a firsthour) are utilized in some descriptions throughout FIGS. 6A-6C and 7A-7Bfor simplicity, the onset times can be elapsed times, representing timessince a previous watermark outage.

Similarly, the outage duration plot 700B includes a seventh onset time716 and an eighth onset time 718, corresponding to two separate onsettimes that help explain the different durations represented by the thirdonset time 614. In this example, the seventh onset time 716 correspondsto an onset time of fifty-five minutes occurring in a first hour, andthe eighth onset time 718 corresponds to an onset time of fifty-fiveminutes occurring in a second, consecutive, hour. The eighth onset time718 can also be referred to as being one-hundred-and-fifteen minutes ina two-hour programming period. With this reorganization, the watermarkoutages of the seventh onset time 716 have durations corresponding tothe fourth cluster 708, and the watermark outages of the eighth onsettime 718 have durations corresponding to the fifth cluster 710.

FIG. 8 is a block diagram of an example processor platform 800structured to execute the instructions of FIGS. 3A, 3B, and 4 toimplement the watermark outage monitor 114 of FIGS. 1 and 2. Theprocessor platform 800 can be, for example, a server, a personalcomputer, a workstation, a self-learning machine (e.g., a neuralnetwork), a mobile device (e.g., a cell phone, a smart phone, a tabletsuch as an iPad™), a personal digital assistant (PDA), an Internetappliance, a DVD player, a CD player, a digital video recorder, aBlu-ray player, a gaming console, a personal video recorder, a set topbox, a headset or other wearable device, or any other type of computingdevice.

The processor platform 800 of the illustrated example includes aprocessor 812. The processor 812 of the illustrated example is hardware.For example, the processor 812 can be implemented by one or moreintegrated circuits, logic circuits, microprocessors, GPUs, DSPs, orcontrollers from any desired family or manufacturer. The hardwareprocessor may be a semiconductor based (e.g., silicon based) device. Inthis example, the processor implements the example media receiver 202,the example watermark detector 204, the example outage analyzer 206, theexample sequence analyzer 208, the example sequence model manager 210,the example sequence model evaluator 212, the example grouping analyzer214, the example grouping model manager 216, the example grouping modelevaluator 218, the example alert generator 220 and/or, more generally,the example watermark outage monitor 114 of FIG. 1.

The processor 812 of the illustrated example includes a local memory 813(e.g., a cache). The processor 812 of the illustrated example is incommunication with a main memory including a volatile memory 814 and anon-volatile memory 816 via a bus 818. The volatile memory 814 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS® Dynamic Random Access Memory(RDRAM®) and/or any other type of random access memory device. Thenon-volatile memory 816 may be implemented by flash memory and/or anyother desired type of memory device. Access to the main memory 814, 816is controlled by a memory controller.

The processor platform 800 of the illustrated example also includes aninterface circuit 820. The interface circuit 820 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), a Bluetooth® interface, a near fieldcommunication (NFC) interface, and/or a PCI express interface.

In the illustrated example, one or more input devices 822 are connectedto the interface circuit 820. The input device(s) 822 permit(s) a userto enter data and/or commands into the processor 812. The inputdevice(s) can be implemented by, for example, an audio sensor, amicrophone, a camera (still or video), a keyboard, a button, a mouse, atouchscreen, a track-pad, a trackball, isopoint and/or a voicerecognition system.

One or more output devices 824 are also connected to the interfacecircuit 820 of the illustrated example. The output devices 1024 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay (LCD), a cathode ray tube display (CRT), an in-place switching(IPS) display, a touchscreen, etc.), a tactile output device, a printerand/or speaker. The interface circuit 820 of the illustrated example,thus, typically includes a graphics driver card, a graphics driver chipand/or a graphics driver processor.

The interface circuit 820 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem, a residential gateway, a wireless access point, and/or a networkinterface to facilitate exchange of data with external machines (e.g.,computing devices of any kind) via a network 826. The communication canbe via, for example, an Ethernet connection, a digital subscriber line(DSL) connection, a telephone line connection, a coaxial cable system, asatellite system, a line-of-site wireless system, a cellular telephonesystem, etc.

The processor platform 800 of the illustrated example also includes oneor more mass storage devices 828 for storing software and/or data.Examples of such mass storage devices 828 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, redundantarray of independent disks (RAID) systems, and digital versatile disk(DVD) drives.

The machine executable instructions 832 of FIGS. 3A, 3B, and 4 may bestored in the mass storage device 828, in the volatile memory 814, inthe non-volatile memory 816, and/or on a removable non-transitorycomputer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods,apparatus and articles of manufacture have been disclosed that enableaccurate watermark outage alert issuance by generating and updatingmodels in association with parameters of watermark outages. Examplesdisclosed herein enable accurate watermark outage alerts by comparingonset times and/or durations associated with detected watermark outageswith models associated with these parameters to determine whether thedetected watermark outage is an expected outage. Such techniques reducethe likelihood of unnecessary watermark outage alerts (e.g., falsealarms), which are prevalent in conventional approaches that may issuewatermark outages during commercials, programming breaks, or otherintentionally non-watermarked portions of media signals.

Although certain example methods, apparatus and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. An apparatus comprising: at least one memory;instructions; and processor circuitry to execute the instructions to atleast: evaluate at least one of an onset time or a duration of adetected watermark outage based on at least one model representative ofexpected watermark outages to determine whether the detected watermarkoutage corresponds to at least one of the expected watermark outagesrepresented in the at least one model; generate an alert in response toa determination that the detected watermark outage does not correspondto any of the expected watermark outages represented in the at least onemodel; and suppress the alert in response to a determination that thedetected watermark outage corresponds to a first one of the expectedwatermark outages represented in the at least one model.
 2. Theapparatus of claim 1, wherein the at least one model includes at leastone of a first model representative of the expected watermark outages ora second model representative of the expected watermark outages, thefirst model to represent expected sequences of the expected watermarkoutages, the second model to represent expected clusters of the expectedwatermark outages.
 3. The apparatus of claim 1, wherein the processorcircuitry is to reset a watermark encoder in response to thedetermination that the detected watermark outage does not correspond toany of the expected watermark outages represented in the at least onemodel.
 4. The apparatus of claim 2, wherein the first model includes:states representative of onset times of respective ones of the expectedwatermark outages represented in the first model; and transitionsbetween respective pairs of the states, respective ones of thetransitions associated with corresponding probabilities of occurrence ofpairs of previous watermark outages and subsequent watermark outagesrepresented by the respective pairs of the states.
 5. The apparatus ofclaim 2, wherein to evaluate the at least one of the onset time or theduration of the detected watermark outage, the processor circuitry isto: determine whether the onset of the detected watermark outagecorresponds to any of the clusters included in the second model; and inresponse to a determination that the onset of the detected watermarkoutage corresponds to a first one of the clusters, determine whether theduration of the detected watermark outage falls into the first one ofthe clusters.
 6. The apparatus of claim 4, wherein the states arerepresentative of the onset times and durations of the respective onesof the expected watermark outages represented in the first model.
 7. Theapparatus of claim 4, wherein the detected watermark outage is a seconddetected watermark outage, the onset time of the detected watermarkoutage is a second onset time of the second detected watermark outage,and to evaluate the second onset time of the second detected watermarkoutage, the processor circuitry is to determine whether the second onsettime of the second detected watermark outage and a first onset time of apreceding first detected watermark outage correspond to a transitionbetween a first state and a second state included in the first model,the first state representative of the first onset time and the secondstate representative of the second onset time.
 8. At least onenon-transitory computer readable medium comprising computer readableinstructions that, when executed, cause at least one processor to atleast: evaluate at least one of an onset time or a duration of adetected watermark outage based on at least one model representative ofexpected watermark outages to determine whether the detected watermarkoutage corresponds to at least one of the expected watermark outagesrepresented in the at least one model; generate an alert in response toa determination that the detected watermark outage does not correspondto any of the expected watermark outages represented in the at least onemodel; and suppress the alert in response to a determination that thedetected watermark outage corresponds to a first one of the expectedwatermark outages represented in the at least one model.
 9. The at leastone non-transitory computer readable medium of claim 8, wherein the atleast one model includes at least one of a first model representative ofthe expected watermark outages or a second model representative of theexpected watermark outages, the first model to represent expectedsequences of the expected watermark outages, the second model torepresent expected clusters of the expected watermark outages.
 10. Theat least one non-transitory computer readable medium of claim 8, whereinthe instructions are to cause the at least one processor to reset awatermark encoder in response to the determination that the detectedwatermark outage does not correspond to any of the expected watermarkoutages represented in the at least one model.
 11. The at least onenon-transitory computer readable medium of claim 9, wherein the firstmodel includes: states representative of onset times of respective onesof the expected watermark outages represented in the first model; andtransitions between respective pairs of the states, respective ones ofthe transitions associated with corresponding probabilities ofoccurrence of pairs of previous watermark outages and subsequentwatermark outages represented by the respective pairs of the states. 12.The at least one non-transitory computer readable medium of claim 9,wherein to evaluate the at least one of the onset time or the durationof the detected watermark outage, the instructions are to cause the atleast one processor to: determine whether the onset of the detectedwatermark outage corresponds to any of the clusters included in thesecond model; and in response to a determination that the onset of thedetected watermark outage corresponds to a first one of the clusters,determine whether the duration of the detected watermark outage fallsinto the first one of the clusters.
 13. The at least one non-transitorycomputer readable medium of claim 11, wherein the states arerepresentative of the onset times and durations of the respective onesof the expected watermark outages represented in the first model. 14.The at least one non-transitory computer readable medium of claim 11,wherein the detected watermark outage is a second detected watermarkoutage, the onset time of the detected watermark outage is a secondonset time of the second detected watermark outage, and to evaluate thesecond onset time of the second detected watermark outage, theinstructions are to cause the at least one processor to determinewhether the second onset time of the second detected watermark outageand a first onset time of a preceding first detected watermark outagecorrespond to a transition between a first state and a second stateincluded in the first model, the first state representative of the firstonset time and the second state representative of the second onset time.15. An apparatus comprising: at least one memory; instructions; andprocessor circuitry to execute the instructions to at least: determinewhether a first onset time of a first detected watermark outage includedin watermark outage data and a second onset time of a preceding seconddetected watermark outage included in the watermark outage datacorrespond to at least one state transition included in a modelrepresentative of expected watermark outages; and in response to adetermination that the first onset time of the first detected watermarkoutage and the second onset time of the second detected watermark outagecorrespond to a first state transition between a first state and asecond state included in the model, associate the first detectedwatermark outage with the first state and update one or more valuesassociated with a corresponding one or more state transitions includedin the model.
 16. The apparatus of claim 15, wherein the model includes:a plurality of states representative of onset times of respective onesof the expected watermark outages represented in the model, theplurality of states including the first state and the second state, thefirst state associated with the first onset time, the second stateassociated with the second onset time; and a plurality of statetransitions between respective pairs of the plurality of states,respective ones of the transitions associated with correspondingprobabilities of occurrence of pairs of previous watermark outages andsubsequent watermark outages represented by the respective pairs of thestates.
 17. The apparatus of claim 15, wherein the processor circuitryis to, in response to a determination that the first onset time of thefirst detected watermark outage and the second onset time of the seconddetected watermark outage do not correspond to any state transitionincluded in the model: add a new state representative of the first onsettime of the first detected watermark outage to the model; and add a newstate transition to the model, the new state transition from an existingstate representative of the second onset time to the new staterepresentative of the first onset time.
 18. The apparatus of claim 15,wherein the processor circuitry is to: evaluate a sliding time windowthat defines a first portion of the watermark outage data to be utilizedto generate the model; and update the model to exclude contributionsfrom a second portion of the watermark outage data that is outside thesliding time window.
 19. The apparatus of claim 15, wherein the model isa first model, and the processor circuitry is to: determine whether athird onset time for a third detected watermark outage in the watermarkoutage data corresponds to at least one cluster in a second modelrepresentative of expected watermark outages; and in response to adetermination that the third detected watermark outage is associatedwith a first cluster in the second model, determine whether a durationof the third watermark outage falls into the first cluster.
 20. Theapparatus of claim 19, wherein the processor circuitry is to: associatethe third detected watermark outage with the first cluster in responseto a determination that the duration of the third detected watermarkoutage falls into the first cluster; and generate a new cluster toinclude in the second model in response to a determination that theduration of the third detected watermark outage does not falls into thefirst cluster, the new cluster corresponding to the third onset time andthe duration of the third detected watermark outage.